Solving a Class of Cut-Generating Linear Programs via Machine Learning

Cut-generating linear programs (CGLPs) play a key role as a separation oracle to produce valid inequalities for the feasible region of mixed-integer programs. When incorporated inside branch-and-bound, the cutting planes obtained from CGLPs help to tighten relaxations and improve dual bounds. However, running the CGLPs at the nodes of the branch-and-bound tree is computationally cumbersome due to the large number of node candidates and the lack of a priori knowledge on which nodes admit useful cutting planes. As a result, CGLPs are often avoided at default settings of branch-and-cut algorithms despite their potential impact on improving dual bounds. In this paper, we propose a novel framework based on machine learning to approximate the optimal value of a CGLP class that determines whether a cutting plane can be generated at a node of the branch-and-bound tree. Translating the CGLP as an indicator function of the objective function vector, we show that it can be approximated through conventional data classification techniques. We provide a systematic procedure to efficiently generate training data sets for the corresponding classification problem based on the CGLP structure. We conduct computational experiments on benchmark instances using classification methods such as logistic regression. These results suggest that the approximate CGLP obtained from classification can improve the solution time compared to that of conventional cutting plane methods. Our proposed framework can be efficiently applied to a large number of nodes in the branch-and-bound tree to identify the best candidates for adding a cut

Consistency in integer programming

In this thesis, we study the role of cutting planes in reducing the size of the branch-and-bound tree for integer programs. Cutting planes are traditionally used to tighten relaxations of the problem by excluding fractional solutions, which leads to improving the dual bounds. In this research, we study a fundamentally different role of cutting planes that target excluding integer infeasible partial solutions, which leads to improving consistency of the set. Consistency helps to reduce backtracking, thereby reducing the size of the search tree. We investigate the connections between consistency and convex hull, and design a cutting plane framework to achieve consistency. One of the practical challenges in incorporating cutting planes inside of branch-and-bound is the running time required to solve a cut-generating problem. To address this difficulty, we design a function approximation framework to find an estimate for the cut-generating functions. We use machine learning methods that provide a classifier for the binary outcome of the indicator function representing the cut-generating oracle. Computational experiments conducted show significant time improvement as well as size reduction of B&B tree for the proposed approach compared to traditional technique

Consistency in integer programming

In this thesis, we study the role of cutting planes in reducing the size of the branch-and-bound tree for integer programs. Cutting planes are traditionally used to tighten relaxations of the problem by excluding fractional solutions, which leads to improving the dual bounds. In this research, we study a fundamentally different role of cutting planes that target excluding integer infeasible partial solutions, which leads to improving consistency of the set. Consistency helps to reduce backtracking, thereby reducing the size of the search tree. We investigate the connections between consistency and convex hull, and design a cutting plane framework to achieve consistency. One of the practical challenges in incorporating cutting planes inside of branch-and-bound is the running time required to solve a cut-generating problem. To address this difficulty, we design a function approximation framework to find an estimate for the cut-generating functions. We use machine learning methods that provide a classifier for the binary outcome of the indicator function representing the cut-generating oracle. Computational experiments conducted show significant time improvement as well as size reduction of B&B tree for the proposed approach compared to traditional technique

Consistency in integer programming

In this thesis, we study the role of cutting planes in reducing the size of the branch-and-bound tree for integer programs. Cutting planes are traditionally used to tighten relaxations of the problem by excluding fractional solutions, which leads to improving the dual bounds. In this research, we study a fundamentally different role of cutting planes that target excluding integer infeasible partial solutions, which leads to improving consistency of the set. Consistency helps to reduce backtracking, thereby reducing the size of the search tree. We investigate the connections between consistency and convex hull, and design a cutting plane framework to achieve consistency. One of the practical challenges in incorporating cutting planes inside of branch-and-bound is the running time required to solve a cut-generating problem. To address this difficulty, we design a function approximation framework to find an estimate for the cut-generating functions. We use machine learning methods that provide a classifier for the binary outcome of the indicator function representing the cut-generating oracle. Computational experiments conducted show significant time improvement as well as size reduction of B&B tree for the proposed approach compared to traditional technique

Consistency in integer programming

In this thesis, we study the role of cutting planes in reducing the size of the branch-and-bound tree for integer programs. Cutting planes are traditionally used to tighten relaxations of the problem by excluding fractional solutions, which leads to improving the dual bounds. In this research, we study a fundamentally different role of cutting planes that target excluding integer infeasible partial solutions, which leads to improving consistency of the set. Consistency helps to reduce backtracking, thereby reducing the size of the search tree. We investigate the connections between consistency and convex hull, and design a cutting plane framework to achieve consistency. One of the practical challenges in incorporating cutting planes inside of branch-and-bound is the running time required to solve a cut-generating problem. To address this difficulty, we design a function approximation framework to find an estimate for the cut-generating functions. We use machine learning methods that provide a classifier for the binary outcome of the indicator function representing the cut-generating oracle. Computational experiments conducted show significant time improvement as well as size reduction of B&B tree for the proposed approach compared to traditional technique

Solving Cut-Generating Linear Programs via Machine Learning

Cut-generating linear programs (CGLPs) play a key role as a separation oracle to produce valid inequalities for the feasible region of optimization problems. When incorporated inside of branch-and-bound, the cutting planes obtained from CGLPs help to tighten relaxations and improve dual bounds. Running CGLPs at nodes of the branch-and-bound tree, however, is computationally cumbersome due to the large number of node candidates and the lack of a priori knowledge on which nodes admit useful cutting planes. As a result, CGLPs are often avoided at default settings of branch-and-cut algorithms despite their potential impact on improving dual bounds. In this paper, we propose a novel framework based on machine learning to approximate the optimal value of the CGLP, which is the deciding factor in generating cutting planes. Translating the CGLP as an indicator function of the objective function vector, we show that it can be approximated through conventional data classification techniques. We provide a systematic procedure to efficiently generate train data sets for the corresponding classification problem based on the CGLP structure. We conduct computational experiments using classification methods such as logistic regression, support vector machines, and neural networks. Computational results suggest that the outcome of the approximate CGLP obtained from classification achieves a high accuracy rate in a significantly smaller amount of time compared to modern LP solvers. Our pro- posed framework can be efficiently applied to a large number of nodes in the branch-and-bound tree to identify the best candidates for running the CGLP--a feature that can be implemented at the preprocessing phase of any branch-and-cut algorithm to improve solution time and bound quality.This is a pre-print of the article Rajabalizadeh, Atefeh, and Danial Davarnia. "Solving Cut-Generating Linear Programs via Machine Learning." (2021). Posted with permission.</p

Achieving Consistency with Cutting Planes

The primary role of cutting planes is to separate fractional solutions of the linear programming relaxation, which results in tighter bounds for pruning the search tree and reducing its size. Bounding, however, has an indirect impact on the size of the search tree. Cutting planes can also reduce backtracking by excluding inconsistent partial assignments that occur in the course of branching, which directly reduces the tree size. A partial assignment is inconsistent with a constraint set when it cannot be extended to a full feasible assignment. The constraint programming community has studied consistency extensively and used it as an effective tool for the reduction of backtracking. We extend this approach to integer programming by defining concepts of consistency that are useful in a branch-and-bound context. We present a theoretical framework for studying these concepts, their connection with the convex hull and their power to exclude infeasible partial assignments. We introduce a new class of cutting planes that target achieving consistency rather than improving dual bounds. Computational experiments on both synthetic and benchmark instances show that the new class of cutting planes can significantly outperform classical cutting planes, such as disjunctive cuts, by reducing the size of the search tree and the solution time. More broadly, we suggest that consistency concepts offer a new perspective on integer programming that can lead to a better understanding of what makes cutting planes work when used in branch-and-bound search.This is a manuscript of the article Davarnia, Danial, Atefeh Rajabalizadeh, and John Hooker. "Achieving Consistency with Cutting Planes." Mathematical Programming (2022).</p

In-depth Evaluation of APACHE Scoring System Using eICU Database

Due to the importance of intensive care units (ICUs) in hospitals, many researchers have developed new predictive models to assess the patients’ illness severity and estimate their mortality in ICUs. In this study, we assess the predictive power of the Acute Physiology and Chronic Health Evaluation IV (APACHE IV) used in the eICU database. Based on the performed explanatory analysis, we found that APACHE IV is not predictive for all diseases. Receiver operating characteristic (ROC) curve analysis showed that, in some diseases, the AUC is as low as 0.75. Moreover, by showing the insufficiency of 0.5 as a threshold, we estimated an optimal threshold for predicted mortality for each disease. Based on the new threshold, the accuracy of APACHE IV was low (approximately 0.7). We further examined the role of APACHE score in predicting the length of stay and ventilation days by calculating the Pearson correlation coefficients (PCCs)

An Exploratory Analysis of Electronic Intensive Care Unit (eICU) Collaborative Research Database

In the present research, different data exploration methods were applied to electronic ICU (eICU) Collaborative Research Database—an ICU database collected from more than 200 hospitals and over 139,000 ICU patients across the United States. In this study, we explore the distribution of the data, including demographics, conditions, and diseases, and identify significant patterns and relationships. Through an exploratory analysis of the data, including the relationships between gender, ethnicity, diseases, and quality of care and mortality rates, remarkable insights were obtained. To the best of our knowledge, this is the first comprehensive exploratory analysis of the eICU database. A deep understanding of the eICU database could help to identify potential areas of improvement and provides the foundation for further predictive and prescriptive analyses of the data with the ultimate goal of improving ICU treatment procedures for future patients